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DETAILED ACTION 

Response to Amendment 

1 . Applicants' response to the last Office Action, filed on October 8 th , 2009 has been 
entered and made of record. 
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Response to Arguments 

2. Applicant's arguments filed October 8 th , 2009 have been fully considered but they 

are not persuasive. 

a. In response to Applicant's remarks that VM does not teach classifier 
detecting object components of different sizes at multiple scales (remarks page 
1 6 paragraph 1 ), it is noted that VM is used to teach detecting of object 
components of different sizes (the typical size of objects are used for 
classification (Page 459, left column, lines 26-29) and since the objects being 
classified are car, truck, motorbike, person (page 459, left column, lines 11-14) 
each of the objects are inherently to have different sizes). Thus, VM teaches 
detecting object components of different sizes. Examiner relies on the teachings 
of Brady for the concept of detecting object of at different scales (Col. 1 1 , lines 
38-50). 
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b. In response to Applicant's remarks that Laumeyer does not teach a 
method adapted for detecting moving and stationary objects from a moving video 
camera, it is noted that Examiner relies upon Laumeyer to teach detection of 
stationary object from a moving video camera (Fig. 3B and (Paragraph [0059], 
lines 1-2)). Examiner further relies upon VM to teach detection of a moving 
object (page 459, left-hand column, lines 10-14). Therefore, it would been 
obvious for one ordinarily skilled in the art at the time the invention was made to 
combine VM with Laumeyer in order to allow individual vehicles to monitor traffic 
outside their vehicles since the environment outside the vehicle usually contains 
both moving objects such as cars and stationary objects such as road signs. 

c. In response to Applicant's remarks that Laumeyer does not teach 
overlapping component classifiers as defined by claim 105. It is noted that 
Laumeyer teaches the use of color and morphology (shape) to segment or detect 
road signs (Paragraph [0056]). Examiner interprets "overlapping component 
classifiers" to mean classifiers used to determine a single object. In this case, 
Laumeyer's teachings of shape and color are both to use to determine a road 
sign. 
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d. In response to Applicant's remarks that Brady does not teach detecting 
object components of different sizes at multiple scales. It is noted that Examiner 
relies on the teachings of VM for detecting object components of different sizes. 
Examiner relies upon the teachings of Brady to disclose detecting object 
componets at multiple scales. Brady teaches when the tracked moving object 
moves away from the camera it "shrinks" and by shrinking an object it is scaling 
down an object. Thus, Brady reads on the claimed multiple scales tracking. 
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Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-29, 33-45, 47, 49-51, 53, 55-80, 84-94 and 105 are rejected under 35 
U.S.C. 1 03(a) as being unpatentable over VM in view of Laumeyer et al. (US 
2007/0154067 hereinafter "Laumeyer") further in view of Brady et al. (5,761,326 
hereinafter "Brady"). 

As to claim 1 , VM teaches method for detecting one or more objects belonging to 
the same object class comprising the steps of: a), receiving a video sequence from a 
video camera (each camera is attached to a processing unit using a video capture card 
that hosts a set of image video object kernel; page 457, left-hand column, lines 25-28) 
comprised of a plurality of image frames (all video streams consists of frames); b) 
applying one or more classifiers to detect components of objects in an image frame in 
the video sequence (each scene may have one of a number of classifications; page 
459, left-hand column, lines 11-14), wherein the component classifiers include 
classifiers for detecting object components of different sizes (classification methods are 
based on matching of the observed size of the mobile object with previously gathered 
information of typical sizes for object of all classes; page 459, left-hand column, lines 
26-29); c) computing a confidence score based in part on the response from the one or 
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more component detectors (each classification has a confidence factor between 0 and 
1 , 1 being maximum confidence and 0 means no confidence; page 459, left-hand 
column, lines 15-17); d) repeating steps b) and c) to detect components of objects 
belonging to the same object class in additional images frames in the video sequence 
(page 459, left-hand column, lines 8-28); and e) accumulating confidence scores from 
the component detectors to determine if an object is detected (page 459, left-hand 
column, lines 8-37); VM teaches detection of moving objects in a traffic surveillance 
system (page 459, left-hand column, lines 10-14). 

Laumeyer teaches a method for identifying objects depicted in a video stream 
(Title) wherein the cameras collecting the video data is mounted on a vehicle (Fig. 3B) 
and the objects being detected are road signs (Paragraph [0059], lines 1-2). Thus 
Laumeyer's vehicle mounted cameras that detect road signs and pedestrian reads on 
the claimed moving video camera that's capable of detecting stationary objects. 
Therefore, it would have been obvious for one ordinarily skilled in the art at the time the 
invention was made to combine VM's traffic monitoring method with Laumeyer's vehicle 
cameras in order to allow individual vehicles to monitor traffic outside their vehicles 
since the environment outside the vehicle usually contains both moving objects such as 
cars and stationary objects such as road signs. 
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Brady teaches a method for traffic detection, monitoring, management and 
vehicle classification and tracking (Col. 1, lines 12-16), wherein a vehicle is tracked as 
the vehicle is moving as it is moving away from the detection camera the vehicle is 
"shrinking", or the vehicle is tracked in different scales (Col. 1 1 , lines 38-50). Thus, 
Brady's tracking method reads on the claimed multiple scales of the detecting object. 
Therefore, it would have been obvious for one ordinarily skilled in the art at the time the 
invention was made to combine the traffic monitoring and classification methods of VM 
and Laumeyer with the shrinking tracking and classification method since as an object 
moves away from the camera it appears smaller and thus it is a logical way of tracking a 
moving object. 
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As to claim 2, VM teaches wherein if accumulated confidence scores indicate 
high confidence of a presence of an object, the method further comprising the step of: 
identifying the detected components to be an object of a particular type (page 459, left- 
hand column lines 22-25). 

As to claim 3, VM teaches wherein the object class is a vehicle (page 459, left- 
hand column lines 12-14). 

As to claim 4, VM teaches if an object is detected, outputting a detection signal 
and object position (page 459, left-hand column lines 43-58). 

As to claim 5, VM teaches testing geometry constraints on a spatial arrangement 
of detected components in an image; and applying whole-appearance classifiers an 
image patch that contains the detected components and which is aligned according to 
the position of the detected components (page 459, left-hand column lines 26-29). 

As to claim 6, VM teaches wherein the geometry constraints are derived from 
camera parameters (page 459, left-hand column lines 20-21). 

As to claim 7, VM teaches wherein the geometry constraints are derived from 
object size (page 459, left-hand column line 27). 
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As to claim 8, VM teaches wherein the geometry constraints are derived from a 
location of an object appearance in the image (page 459, left-hand column line 46). 

As to claim 9, VM teaches wherein the whole appearance classifiers detect entire 
or partial object appearance, the entire or partial object appearance being aligned 
according to positioning of at least two components (page 459, left-hand column lines 
26-29). 

As to claim 10, VM teaches wherein the component classifiers include classifiers 
for detecting components at multiple scales (page 459, left-hand column lines 30-36). 

As to claim 1 1 , VM teaches wherein component classifiers are defined by 
discriminate features and decision rules which are learned through boosted training 
(page 459, left-hand column lines 37-38). 

As to claim 12, VM teaches wherein the discriminant features include corners 
(page 461 , fig. 4 shows an object that's been identified and the system forming a box 
around the object, the box has corners). 

As to claim 13, VM teaches wherein the discriminant features include horizontal 
edges (page 461 , fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box has horizontal edges). 
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As to claim 14, VM teaches wherein the discriminant features include vertical 
edges (page 461 , fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box has vertical edges). 

As to claim 15, VM teaches wherein the discriminant features include horizontal 
stripes (page 461, fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box may contain horizontal stripes). 

As to claim 16, VM teaches wherein the discriminant features include vertical 
stripes (page 461 , fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box may contain vertical stripes). 

As to claim 17, VM teaches wherein the discriminant features include diagonal 
stripes (page 461 , fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box may contain diagonal stripes). 

As to claim 18, Laumeyer teaches performing an online adaptation to adapt a 
classifier structure to an online pattern (the method of rapidly identifying, locating, and 
storing images of objects from the frames maybe implemented using the internet; 
[0012], lines 1-6, 9-10). 
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As to claim 19, Laumeyer teaches wherein the step of performing an online 
adaptation further comprises the step of: applying a dynamic switching strategy to direct 
the detector to take appropriate weak classifiers as discriminants according to auxiliary 
information about the online pattern (selected portions of individual frames are used to 
confirm that a set of several images in fact represents a single object of a class of 
objects; [0012], lines 11-24). 

As to claim 20, VM teaches wherein the one or more classifiers include 
overlapping component classifiers (page 461 , fig. 4 shows an object that's been 
identified and the system forming a box around the object; this box is formed by first 
gathering the structure of the object being classified then the corresponding shape is 
matched with the 2D mask to yield a scaled information. From the 3D points 
information such as size, position and speed maybe extracted (Page 462 left-hand 
column lines 3-8). The overlapping component may be the different information 
extracted from the 3D information, since all information is based on a single object). 

As to claim 21 , VM teaches wherein the overlapping component classifiers 
comprises four corners representing a rear profile of a vehicle (page 461 , fig. 4 shows 
an object that's been identified and the system forming a box around the object, where 
the object is a vehicle driving on a road). 
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As to claim 22, VM teaches wherein the overlapping component classifiers 
comprises four corners representing a frontal profile of a vehicle (page 461 , fig. 4 shows 
an object that's been identified and the system forming a box around the object, where 
the object is a vehicle driving on a road). 

As to claim 23, VM teaches wherein the overlapping component classifiers 
comprises four corners representing a side profile of a vehicle (page 461 , fig. 4 shows 
an object that's been identified and the system forming a box around the object, where 
the object is a vehicle driving on a road). 

As to claim 24, VM teaches wherein one of the overlapping component classifiers 
detects the bottom left corner of a vehicle (page 461 , fig. 4 shows an object that's been 
identified and the system forming a box around the object, where the object is a vehicle 
driving on a road). 

As to claim 25, VM teaches wherein one of the overlapping component 
classifiers detects the bottom right corner of a vehicle (page 461 , fig. 4 shows an object 
that's been identified and the system forming a box around the object, where the object 
is a vehicle driving on a road). 
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As to claim 26, VM teaches wherein one of the overlapping component classifiers 
detects the top left corner of a vehicle (page 461 , fig. 4 shows an object that's been 
identified and the system forming a box around the object, where the object is a vehicle 
driving on a road). 

As to claim 27, VM teaches wherein one of the overlapping component classifiers 
detects the top right corner of a vehicle (page 461 , fig. 4 shows an object that's been 
identified and the system forming a box around the object, where the object is a vehicle 
driving on a road). 

As to claim 28, VM teaches wherein positioning of the four corners of the rear 
profile for a vehicle differ for different classes of vehicles (page 461 , fig. 4 shows an 
object that's been identified and the system forming a box around the object, where the 
object is a vehicle driving on a road). 

As to claim 29, VM teaches a class of vehicle includes sedans (car; page 459, 
left-hand column lines 12-14). 

As to claim 33, VM teaches wherein a class of vehicle includes trucks (truck; 
page 459, left-hand column lines 12-14). 
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As to claim 34, VM teaches wherein a distance between any two corners of the 
vehicle is constrained (page 461 , fig. 4 shows an object that's been identified and the 
system forming a box around the object, where the object is a vehicle driving on a road). 

As to claim 35, VM teaches wherein the constraint between any two corners of 
the vehicle is scaled based on a distance between the vehicle and a camera capturing 
the video sequence and camera parameters (page 459, left-hand column lines 18-21). 

As to claim 36, VM does not teach wherein an image pyramid of multiple 
resolutions is used to detect objects of size 2*x, 4*x and so on with the classifier for the 
size x. Brady teaches a method for machine vision classification and tracking using 
multiple resolution of a region of interest that has a sample kernel of 2x2, even though 
this method only give one sample it is not limited to kernel size 2x2 (Col. 12, lines 34- 
45). Thus, Brady's machine vision method reads on the claimed pyramid multiple 
resolution method to detect objects. Therefore, it would have been obvious to one 
ordinarily skilled in the art at the time the invention was made to combine VM's 
surveillance system with Brady's multiple resolution system in order to increase the 
speed of processing the information for tracking (Brady Col. 12, lines 34-35). 
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As to claim 37, VM teaches wherein the accumulated confidence scores are 
inferred from confidence scores across multiple frames using a recursive filter (the 
algorithm used to determine the next trajectory adjusts itself whenever new information 
is received, thus making it repeat and re-occur; page 459, left-hand column lines 49-50; 
page 459, right-hand column lines 1-2). 

As to claim 38, VM teaches wherein when the accumulated confidence score is a 
linear combination of the confidence scores of multiple component classifiers and the 
whole-appearance classifiers (page 459, left-hand column lines 22-25). 

As to claim 39, VM teaches wherein when the confidence score for a principal 
component classifier is sufficiently high, the confidence score of the remaining 
component classifiers and the whole-appearance classifier are computed (page 459, 
left-hand column lines 15-21). 

As to claim 40, VM teaches wherein if an object is detected the method 
comprising the step of: tracking the object over subsequent image frames (page 459, 
right-hand column lines13-20). 

As to claim 41 , VM teaches wherein the step of tracking the object further 
comprises the step of: restricting an area of search in each subsequent image frame 
based on the location of the object in a current image frame (page 459, right-hand 
column lines 24-31). 
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As to claim 42, VM teaches wherein the step of tracking the object further 
comprises the step of: determining the optimal classifier scale based on a distance 
between the object and a camera detecting the object and camera parameters (page 
459, right-hand column lines 32-41 teaches the prediction process using speed, 
however, speed is the combination of time and distance, therefore there is the need to 
use the distance parameter). 

As to claim 43, VM does not teach wherein the confidence scores of component 
classifiers are computed in a coarse to fine framework. Brady teaches a method for 
machine vision classification and tracking that searches a target region first at a lower 
resolution then at a higher resolution (Col. 12, lines 43-46). Thus, Brady's searching a 
target region from low to high resolution reads on the claimed coarse to fine framework. 
Therefore, it would have been obvious to one ordinarily skilled in the art at the time the 
invention was made to combine VM's surveillance system with Brady's multiple 
resolution system in order to increase the speed of processing the information for 
tracking (Brady Col. 12, lines 34-35). 
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As to claim 44, VM does not teach wherein detection is performed on an image 
pyramid of multiple resolutions. Brady teaches a method for machine vision 
classification and tracking using multiresolution pyramid (Col. 12, lines 41-43). Thus, 
Brady's multiresolution pyramid reads on the claimed image pyramid of multiple 
resolutions. Therefore, it would have been obvious to one ordinarily skilled in the art at 
the time the invention was made to combine VM's surveillance system with Brady's 
multiple resolution system in order to increase the speed of processing the information 
for tracking (Brady Col. 12, lines 34-35). 

As to claim 45, VM teaches wherein an object class includes pedestrians 
(person; page 459, left-hand column lines 12-14). 

As to claim 47, VM teaches wherein an object class includes motorcycles 
(motorbike; page 459, left-hand column lines 12-14). 

As to claims 49-51 , 53, 55, 58-80, 84-85 and 87-92, they are the system claims 
of claims 1 , 3, 45, 47, 2, 5-21 , 24-29, 34-35 and 37-42. Please see above for detail 
analysis. 
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As to claim 56, VM does not teach wherein if an object is detected, the processor 
outputs a warning signal. Brady teaches the graphical user interface automatically 
displays alarm information when an incident has been detected (Col.4, lines 5-7). Thus, 
Brady's alarm information reads on the claimed warning signal. Therefore, it would 
have been obvious to one ordinarily skilled in the art at the time the invention was made 
to combine VM's surveillance system with the alarm notification in order to notify the 
user. 

As to claim 57, VM teaches a display for displaying the video sequences (Fig. 4 
is a sample of a display window of the system). 

As to claims 86 and 93-94, they are the system claims of claims 36 and 43-44. 
Please see above for detail analysis. 

As to claim 105, it is the same as claim 1 with the exception of claim 105 does 
not teach wherein the component classifiers include classifiers for detection 
components at multiple scales, but in stead teaches wherein the one or more classifiers 
include overlapping components classifiers (Laumeyer teaches classifying the object 
detected based on color and shape ([0056], lines 14-18), since the two identifiers are for 
the same object they are considered to be overlapping). 
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5. Claims 30-32, 46, 48, 52, 54, and 81-83 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over VM. 

As to claims 30-32, 46 and 48, VM teaches the classification may have one of 
the following values: car, truck, motorbike, person, toojarge and mobile_object (page 
459, left-hand column lines 10-14). VM does not expressly disclose that the classifier 
also includes bicycles, sports utility vehicles, vans, tractor-trailers and traffic signs. 
However, Examiner takes Official Notice that the non-disclosed classifiers are well 
known in the art. It would have been obvious at the time of the invention was made to 
one of ordinary skill in the art to add the additional classifiers since Examiner takes 
official notice that the non-disclosed classifiers are common on a normal road. 

As to claims 52, 54, and 81-83, they are the system claims of claims 46, 48 and 
30-32. Please see above for detail analysis. 



Application/Control Number: 10/805,946 Page 21 

Art Unit: 2624 

Conclusion 

6. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 
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Contact Information 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CLAIRE WANG whose telephone number is (571)270- 
1051 . The examiner can normally be reached on M-F 9am-6pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Samir Ahmed can be reached on 571-272-7413. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Claire Wang/ 
Examiner, Art Unit 2624 
01/11/2010 



/CHARLES KIM/ 

Primary Examiner, Art Unit 2624 



